Methods for improving the robustness of an automated unloading system

ABSTRACT

System utilizes stereo cameras to sense the position of a tractor-drawn grain cart relative to a combine during the unloading (on-the-go) process. The sensing system also detects the fill level of grain in the cart and adjusts the relative position of the tractor to the combine to achieve an even fill to the desired fill level.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims priority to U.S. Provisional Application Ser.No. 61/814,402, titled METHODS FOR IMPROVING THE ROBUSTNESS OF ANAUTOMATED UNLOADING SYSTEM, filed Apr. 22, 2013, incorporated byreference herein in its entirety.

BACKGROUND OF THE INVENTION

During the unloading process, a tractor operator pulling a grain cartwill generally attempt to match the speed of the combine as it harveststo optimize efficiency. This can sometimes be difficult to do,especially for inexperienced tractor operators. Sometimes, the tractoroperator will set a constant speed (below the optimal speed forharvesting) while unloading, and the harvesting vehicle (combine orforage harvester) operator will adjust the relative position between thevehicles by increasing or decreasing the speed of the combine. Theunloading operation requires much more precision as the grain cartbegins to fill up. The objective is for the tractor and harvestingvehicle operators to choreograph their movements to achieve an even fillof the grain cart. This invention automates this interaction by, forexample, calculating the auger position, relative position of the cartand harvesting vehicle, and point of incidence of the crop into the cartbased on sensing where the grain is being unloaded into the grain cart,sensing and profiling the fill level of the grain cart, and executing astrategy to fill the cart evenly to a desired fill level.

Sometimes, there are circumstances in which it is difficult orimpossible for the system to sense the relative position between theharvesting vehicle and the grain cart, and therefore, where the grain isbeing unloaded into the grain cart. There are several techniques thatcan be utilized to mitigate the negative impact to the system when suchcircumstances arise.

BRIEF DESCRIPTION OF THE DRAWINGS

For the present invention to be easily understood and readily practiced,the invention will now be described, for the purposes of illustrationand not limitation, in conjunction with the following figures, wherein:

FIGS. 1A and 1B are front views of a combine and a forge harvester,respectively, illustrating many features of the present invention;

FIG. 2 is a perspective view of a combine unloading grain into a cart;

FIG. 3 is a side view of a combine illustrating the repositioning of afirst imaging device as an auger moves up and down;

FIG. 4 is a side view of a tractor pulling a cart;

FIG. 5 is a process flow diagram of one embodiment of the presentinvention for adjusting relative offset between the combine and graincart;

FIG. 6 is a process flow diagram of one embodiment of the presentinvention for overcoming conditions that diminish system performance;

FIG. 7 is a top view of the cart illustrating fill zones of one fillstrategy and auger rotational limits;

FIG. 7A is a rear view of a combine alongside a cart illustratingminimum distances and auger angle to avoid contact of the auger with thetop edge of the cart;

FIG. 8 is a block diagram of one embodiment of a stereo vision systemfor a harvesting vehicle for managing the unloading of agriculturalmaterial from the harvesting vehicle (e.g., combine);

FIG. 9 is a block diagram of another embodiment of a stereo visionsystem for a harvesting vehicle for managing the unloading ofagricultural material from the harvesting vehicle (e.g., aself-propelled forage harvester);

FIG. 10 is a block diagram of an embodiment of a system for a receivingvehicle (without stereo vision) for managing the unloading ofagricultural material from a transferring vehicle; and

FIG. 11 is a process flow diagram of Machine Sync logic to control therelative position between the combine and grain cart.

DETAILED DESCRIPTION OF THE INVENTION

In accordance with one embodiment, FIG. 8 shows a system 11 for aharvesting vehicle for managing the unloading of agricultural materialfrom the harvesting vehicle (also referred to herein as a transferringvehicle, a combine, a harvester, and a self-propelled forage harvester)to a receiving vehicle (also referred to herein as a tractor, propelledportion, trailer, grain cart, cart, storage portion, container, orwagon). In one embodiment, the system 11 comprises a first imagingdevice 10 and second imaging device 12 coupled to an image processingmodule 18. The first imaging device 10 may comprise a primary stereocamera, while the second imaging device 12 may comprise a secondarystereo camera. For example, the first imaging device 10 or the secondimaging device 12 is mounted at sufficiently high elevation above groundlevel to have some visibility into the storage container 4 (e.g., graincart), or sufficient visibility of the interior of the storage container4 and its contents, to determine a profile, distribution or level ofagricultural material (e.g., grain) within a volume or portion of thevolume defined by the container 4.

The image processing module 18 may be coupled, directly or indirectly,to lights 15 on a vehicle (e.g., harvesting vehicle) for illumination ofa storage container and/or spout discharge end (e.g., 13A in FIG. 1), orfor illumination of a field of view of the first imaging device 10, thesecond imaging device 12, or both for acquiring raw images (e.g., ofsufficient brightness, contrast and color reproduction). For example,the image processing module 18 may control drivers or switches, which inturn control the activation or deactivation of lights 15 on theharvesting vehicle. The image processing module 18 may activate thelights 15 on the vehicle for illumination of the storage container(e.g., 4 in FIG. 1), spout discharge end (e.g., 13A in FIG. 1) or bothif a light meter indicates that an ambient light level is below acertain minimum threshold. In one configuration, the light metercomprises a photo-sensor, photo-resistor, photo-sensitive device, or acadmium-sulfide cell. The spout also includes a length and a rotationalend pivotally connected to the transferring vehicle allowing the spoutto rotate about a vertical axis of rotation.

In one embodiment, the auger rotation system 16 may comprise: (1) arotation angle sensor for sensing a spout rotation angle (e.g., φ inFIG. 1) or other spout angles of the spout/auger 47 with respect to oneor more axes of rotation and (2) an actuator for moving the spout/auger47 to change the spout rotation angle φ or other spout angles; hence,the spout position with respect to the receiving vehicle 6 or itsstorage container 4. The actuator of the auger rotation system 16 maycomprise one or more motors, a linear motor, an electro-hydraulicdevice, a ratcheting or cable-actuated mechanical device, or anotherdevice for moving the spout 89, or the spout discharge end 87. The spoutangle or spout rotation angle φ may comprise a simple angle, a compoundangle or multi-dimensional angles that are measured with reference toany of the following: a reference axis parallel to the direction oftravel of the harvesting vehicle, a generally vertical axis, a generallyhorizontal axis, or an axis generally orthogonal to at least one of thegenerally vertical axis and the generally horizontal axis.

Where the system 11 of FIG. 8 is applied to a combine or a harvester,the spout/auger 47 may be controlled in one or more dimensions (e.g., ofrotation or movement). In one configuration, the auger rotation system16 (of the harvester or combine) controls a rotation angle φ of thespout/auger 47 in a generally horizontal plane or about a generallyvertical axis. In another configuration, the auger rotation system 16 orspout controller may control one or more of the following angles: (1)rotation angle φ of the spout in a generally horizontal plane, (2) tiltangle in a relatively vertical plane, and (3) flap angle (e.g., forageharvester), where the rotation angle, tilt angle and flap angle areassociated with mutually orthogonal axes. In one configuration, bycontrolling the rotation angle, the vehicle controller (e.g., 46 of FIG.8, 54 of FIG. 9) may automatically extend or retract the spout/auger 47(e.g., unloading auger arm) when appropriate (e.g., when unloading ofthe agricultural material is complete).

The vehicle controller 46 controls the rotation of the auger 47 fortransfer or movement of the agricultural material from the harvestingvehicle 2 to the receiving vehicle 6. The vehicle controller 46 canprovide a data message that indicates when the auger 47 for unloadingagricultural material from the harvesting vehicle is activate andinactive. The auger 47 may comprise an auger, an electric motor fordriving the auger, and a rotation sensor for sensing rotation of theauger or its associated shaft. In one embodiment, the auger 47 isassociated with a container for storing agricultural material (e.g., agrain tank) of a harvesting vehicle 2 (e.g., a combine).

If the image processing module 18 or another sensor determines that thecontainer 4 has reached a target fill level (e.g., full or somepercentage or fraction of capacity), the image processing module 18,vehicle controller 46, or auger rotation system 16 may automaticallyshut off the unloading auger 47.

The imaging processing module 18 may comprise a controller, amicrocomputer, a microprocessor, a microcontroller, an applicationspecific integrated circuit, a programmable logic array, a logic device,an arithmetic logic unit, a digital signal processor, or another dataprocessor and supporting electronic hardware and software.

In one embodiment, the image processing module 18 comprises a containeridentification module 20, and an alignment module 24.

The image processing module 18 may be associated with a data storagedevice 19. The data storage device 19 may comprise electronic memory,non-volatile random access memory, a magnetic disc drive, an opticaldisc drive, a magnetic storage device or an optical storage device, forexample. If the container identification module 20 and the alignmentmodule 24 are software modules they are stored within the data storagedevice 19.

The container identification module 20 identifies a set oftwo-dimensional or three dimensional points (e.g., in Cartesiancoordinates or Polar coordinates) in the real world that define at leasta portion of the container perimeter (e.g., front edge or rear edge) ofthe storage portion (e.g., cart 4 in FIG. 1). The set of two-dimensionalor three dimensional points correspond to pixel positions in imagescollected by the first imaging device 10, the second imaging device 12,or both. The container identification module 20 may use or retrievecontainer reference data.

The container reference data comprises one or more of the following:reference dimensions, reference shape, drawings, models, layout, andconfiguration of the container 4, such as the container perimeter, thecontainer edges; reference dimensions, reference shape, drawings,models, layout, and configuration of the entire storage portion 4 ofreceiving vehicle 6; storage portion wheelbase, storage portion turningradius, storage portion hitch configuration of the storage portion 4 ofthe receiving vehicle 6. The container reference data may be stored andretrieved from the data storage device 19 (e.g., non-volatile electronicmemory). For example, the container reference data may be stored by,retrievable by, or indexed by a corresponding receiving vehicleidentifier in the data storage device 19 of the harvesting vehiclesystem 11. For each receiving vehicle identifier, there can be acorresponding unique container reference data stored therewith in thedata storage device 19.

In one embodiment, the harvesting vehicle 2 receives a data message fromthe receiving vehicle 6 in which a vehicle identifier of the receivingvehicle is regularly (e.g., periodically transmitted). In anotherembodiment, the harvesting vehicle 2 interrogates the receiving vehicle6 for its vehicle identifier or establishes a communications channelbetween the harvesting vehicle 2 and the receiving vehicle 6 inpreparation for unloading via the wireless communication devices (48,148). In yet another embodiment, the receiving vehicle 6 transmits itsvehicle identifier to the harvesting vehicle 2 when the receivingvehicle 6 approaches the harvesting vehicle 2 within a certain radialdistance. In still another embodiment, only one known configuration ofreceiving vehicle is used with a corresponding harvesting vehicle andthe container reference data is stored or saved in the data storagedevice 20. In the latter embodiment, the harvesting vehicle 2 isprogrammed, at least temporarily, solely for receiving vehicles withidentical containers, which are identical in dimensions, capacity,proportion and shape.

If the linear orientation of a set of pixels in the collected image dataconforms to one or more edges of the perimeter of cart 4 as prescribedby the container reference data, the position of the container has beenidentified. A central region or central zone of the container opening ofthe container 4 can be identified by dividing the distance (e.g.,shortest distance or surface normal distance) between opposite sides ofthe container, or by identifying corners of the container and wherediagonal lines that intercept the corners intersect, among otherpossibilities.

The alignment module 24 estimates motion commands at regular intervalsto maintain alignment of the spout discharge end 13A over the target ofthe container 4 for unloading agricultural material. The alignmentmodule 24 may send commands to the harvesting vehicle 2 with respect toits speed, velocity or heading to maintain alignment of the position ofthe harvesting vehicle with respect to the receiving vehicle. Forexample, the alignment module 24 may transmit a steering command orheading command to the steering controller 32, a braking or decelerationcommand to a braking system 34, and a propulsion, acceleration or torquecommand to a propulsion controller 40. Further, similar command data maybe transmitted via the wireless communication devices (48, 148) to thereceiving vehicle 6 for observational purposes or control of thereceiving vehicle via its steering system controller 32, its brakingcontroller 36.

The propulsion controller 40 may facilitate an even distribution ofagricultural material in the container 4 by acceleration or decelerationof the receiving vehicle 6 or its propulsion portion 75 to redistributeevenly or move the agricultural material in the container 4. In oneexample, a propulsion controller 40 temporarily increases its groundspeed, or alternately a relative speed of the receiving vehicle 6relative to the harvesting vehicle 2, if an image processing module 18senses that a front volume of the storage portion is currently filled toa target level (with agricultural material) or until the entire cart 4reaches the target fill level. Conversely, propulsion controller 40temporarily decreases its ground speed, or alternately a relative speedof the receiving vehicle relative to the harvesting vehicle, if an imageprocessing module 18 senses that a rear volume of the storage portion iscurrently filled (with agricultural material) to a target level or untilthe entire storage portion 93 reaches the target fill level.

The image processing module 18 provides image data to a user interfaceprocessing module 26 that provides, directly or indirectly, statusmessage data and performance message data to a user interface 44. Asillustrated in FIG. 8, the image processing module 18 communicates witha vehicle data bus 31 (e.g., Controller Area Network (CAN) data bus).

In one embodiment, a location determining receiver 42, a first wirelesscommunications device 48, a vehicle controller 46, a steering controller32, a braking controller 36, and a propulsion controller 40 are capableof communicating over the vehicle data bus 31. In turn, the steeringcontroller 32 is coupled to a steering system 30 of the harvestingvehicle; the braking controller 37 is coupled to the braking system 34of the harvesting vehicle; and the propulsion controller 40 is coupledto the propulsion system 38 of the harvesting vehicle.

In one configuration, a user interface 44 is arranged for enteringcontainer reference data or dimensional parameters related to thereceiving vehicle. For example, the container reference data ordimensional parameters comprise a distance between a trailer hitch(which interconnects the propulsion unit 75 and the storage portion 93)and front wheel rotational axis of the storage portion 93 of thereceiving vehicle 6.

The system 11 of FIG. 8 is well suited for use on a combine or harvesteras the harvesting vehicle. The system 11 of FIG. 8 may communicate andcooperate with a second system (211 of FIG. 10) on the receiving vehicle6 to coordinate the relative alignment of the harvesting vehicle 2 andthe receiving vehicle 6 during unloading or transferring of materialfrom the harvesting vehicle. Like reference numbers in FIG. 8 and FIG. 9indicate like elements.

The system 111 of FIG. 9 is similar to the system 11 of FIG. 8; exceptthat the system 111 of FIG. 9 further comprises an implement data bus58, a gateway 29, and vehicle controllers 50, 54 coupled to the vehicledata bus 60 for the lights 52 and spout adjuster 56. The vehiclecontroller 50 controls the lights 52; the vehicle controller 54 controlsthe spout adjuster 56 for moving or adjusting the orientation or angleof the spout or auger 47, or its spout discharge end 13B. The spoutadjuster 56 may comprise an actuator for moving or adjusting the spout89 and one or more sensors for measuring the spout angle, orientation,or position of the spout 89. For instance, the spout adjuster 56 or itsactuator may comprise a servo-motor, electric motor, or anelectro-hydraulic mechanism for moving or adjusting the spout 89.

Where the system 111 of FIG. 9 is applied to a self-propelled forageharvester, the vehicle controller 54 and spout adjuster 56 may controlor adjust spout or auger 47 in multiple dimensions, such as two or threedimensions. For example, the vehicle controller 54 or spout controllermay control one or more of the following angles: (1) rotation angle ofthe spout in a generally horizontal plane, (2) tilt angle in arelatively vertical plane, and (3) flap angle, where the rotation angle,tilt angle and flap angle are associated with mutually orthogonal axes.For a forage harvester, the spout 47 (e.g., unloading auger arm) is notusually retracted and the flow of agricultural material from the spout89 is generally continuous during harvesting.

If a container 4 of the receiving vehicle 2 is full (or imminentlyapproaching a full state) with agricultural material (e.g., from atransferring operation), as detected by one or more sensors (e.g., massor optical sensors) on the receiving vehicle 6 for detecting a mass,weight or volume of agricultural material in the container 4; theimaging system 18 of the harvesting vehicle 2 or the sensors of thereceiving vehicle via the wireless communications devices 48, 148 maynotify the operator (of the harvesting vehicle 2) on the user interface44 of the full state, fill state or full condition of the container 4.In response to the full state or near full state (e.g., approximately 90percent or more of full capacity) of the container 4 (e.g., and in thecontext of a forage harvester as the harvesting vehicle 2), the system111 of FIG. 9 may: (1) maintain a last position and orientation of thespout 47 to continue to unload agricultural material in the samelocation, or (2) sweep the spout 47 or spout discharge end 13A back andforth in a continuous or incremental motion to continue to evenlydistribute the material in the container 4

In one configuration, the implement data bus 58 may comprise aController Area Network (CAN) implement data bus. Similarly, the vehicledata bus 60 may comprise a controller area network (CAN) data bus. In analternate embodiment, the implement data bus 58, the vehicle data bus60, or both may comprise an ISO (International Organization forStandardization) data bus or ISOBUS, Ethernet or another data protocolor communications standard.

The gateway 29 supports secure or controlled communications between theimplement data bus 58 and the vehicle data bus 60. The gateway 29comprises a firewall or another security device that may restrict orprevent a network element or device on the implement data bus 58 fromcommunicating (e.g., unauthorized communication) with the vehicle databus 60 or a network element or device on the vehicle data bus 31, unlessthe network element or device on the implement data bus 58 follows acertain security protocol, handshake, password and key, or anothersecurity measure. Further, in one embodiment, the gateway 29 may encryptcommunications to the vehicle data bus 60 and decrypt communicationsfrom the vehicle data bus 60 if a proper encryption key is entered, orif other security measures are satisfied. The gateway may allow networkdevices on the implement data bus 58 that communicate via an openstandard or third party hardware and software suppliers, whereas thenetwork devices on the vehicle data bus 60 are solely provided by themanufacturer.

In FIG. 9, a location determining receiver 42, a user interface 44, auser interface processing module 26, and the gateway 29 are coupled tothe implement data bus 58. Vehicle controllers 50, 54 are coupled to thevehicle data bus 60. In turn, the vehicle controllers 50, 54 arecoupled, directly or indirectly, to lights 15 on the harvesting vehicleand the spout 89 of the harvesting vehicle (e.g., self-propelled forageharvester). Although the system of FIG. 9 is well suited for use orinstallation on a self-propelled forage harvester, the system of FIG. 9may also be applied to combines, harvesters or other heavy equipment.

The system 11 of FIG. 8 and the system 111 of FIG. 9 apply to theharvesting vehicle 2, whereas the system of FIG. 10 applies to thereceiving vehicle 2. Like reference numbers in FIG. 9 and FIG. 10indicate like elements. As previously noted, the harvesting vehicle 2comprises a combine, harvester, self-propelled harvester, vehicle orheavy equipment that collects or harvests material for transfer to thereceiving vehicle. In one embodiment, the receiving vehicle 2 comprisesa propelled portion (e.g., tracker in FIG. 1) and a storage portion(e.g., 4 in FIG. 1) for storing the material transferred from theharvesting vehicle 2. The receiving vehicle 6 may comprise thecombination of a tractor and a grain cart or wagon, where the tractor isan illustrative example of the propelled portion 6 and where the graincart is an illustrative example of the storage portion 4. In oneembodiment, FIG. 10 illustrates a propelled portion (e.g., tractor)without a first imaging device 10 or a second imaging device 12 on thepropelled portion 75. Like reference numbers in FIG. 9 and FIG. 10indicate like elements.

The system 211 of FIG. 10 is similar to the system of FIG. 9, except thesystem of FIG. 10 deletes the first imaging device 10, the secondimaging device 12, the image processing module 18, the user interface44, the user interface processing module 26, the vehicle controllers 50,54, the lights 52 and spout 56 from FIG. 9. The system 211 of FIG. 10comprises a second wireless communications device 148 for communicatingwith the first communications device 48 of FIG. 8 or FIG. 9, forexample. The wireless devices 48, 148 may exchange or communicateposition date, relative position data, command data, or control data forcontrolling, adjusting or coordinating the position and orientation ofthe vehicles; more particularly, the position and the orientation of thespout 47 or spout discharge end 13A over the opening of the container 4.The second wireless communications device 148 is coupled to the vehicledata bus 31. In FIG. 10, the system 211 for a receiving vehicle 6 can beused in conjunction with the system (11 or 111) of the harvestingvehicle 2 of FIG. 8 or FIG. 9.

The image processing module 18 estimate a distance or range from thefirst imaging device 10, the second imaging device 12, or both to thepixels or points lying on the container perimeter or on the containeredge. For example, the image processing module 18 may use the disparitymap or image to estimate a distance or range from the first imagingdevice 10, the second imaging device 12, or both to the pixels or pointslying on the container perimeter 81, the container edges 181, thecontainer opening 83, in the vicinity of any of the foregoing items, orelsewhere.

For example, the system 11,111, 211 executes a container fillingstrategy by changing the relative speed, velocity or acceleration of theharvesting vehicle and receiving vehicle (e.g., through the ISO Class 3interface) to promote even or uniform filling of the container 4. As anillustrative example, for a “front-to-back” fill strategy, the system11, 111, 211 has the receiving vehicle generally maintain a constantfore/aft distance relative to the harvesting vehicle (e.g., combine)such that the agricultural material is filling the front volume of thecart and putting weight on the tongue of the connection between thepropulsion portion 6 and the storage portion. When the front volume ofthe container 4 becomes full or attains a target volume or mass ofagricultural material, the system 11, 111, 211 can command the receivingvehicle to temporarily increase its speed or velocity, or accelerate,relative to the ground (or relative to the harvesting vehicle) via thepropulsion controller 40 so that the agricultural material flowing fromspout 47 drops or moves further toward the rear of the container 4(e.g., from the force of acceleration on the receiving vehicle). Duringor in preparation for the acceleration, if warranted by the estimatedalignment of the spout discharge end 13A and the container perimeter oredges from the alignment module 24 or the image processing module 18,the vehicle controller 46 may suspend rotation temporarily of the auger47 to avoid spilling agricultural material or missing the container 4,or the alignment module 24 provides command data via the wirelesscommunication devices 48, 148 such that the propulsion controller 40 ofthe harvesting vehicle accelerates simultaneously (e.g., with equalmagnitude and direction to the receiving vehicle) to maintain alignment(e.g., substantially the same alignment) between the container perimeteror edge and the spout discharge end 13A. The system 11, 111, 211 orpropulsion controller 40 can temporarily increase a relative speed ofthe receiving vehicle relative to the harvesting vehicle if the imageprocessing module 18 senses that a front volume of the cart 4 iscurrently filled to a target level or until the entire storage portion93 reaches the target fill level. For instance, the system 11, 111, 211can repeat the process of temporarily increasing the speed or velocityof the receiving vehicle relative to the harvesting vehicle during eachsampling interval that the image processing module 18 senses that thefront volume of the container 4 is currently filled to a target level oruntil the entire container 4 reaches the desired fill level.

In another configuration, as the container 4 of the receiving vehiclebegins to become full, the agricultural material inside the container 4was, is or becomes visible to the first imaging device 10, the secondimaging device 12, or both. Next, the imaging device 10, 12 can scan orprofile the height or level of the agricultural material inside thecontainer 4. With an accurate profile of the height of level of theagricultural material inside the container 4, the system can thenexecute a container filling strategy that is appropriate for thesituation.

The system and method is well-suited for controlling the steering andspeed of the harvesting vehicle and the receiving vehicle via locationdetermining receivers and wireless communication devices. Further, thesystem and method facilitates detection of how the container of thereceiving vehicle is being filled to adjust the relative lateralalignment, and fore/aft alignment between the spout 47 or spoutdischarge end 13A and the container perimeter or edge to achieve uniformfilling or uniformly distributed height level of agricultural materialwithin the container 4. Uniform filling of agricultural material withinthe container 4 can be realized to minimize certain errors that mightotherwise result from fatigue, inexperience or skill shortcomings of theoperator of the vehicles, for example.

The above described system architecture and software modules operativelyexecute a system referred to herein as Machine Sync. FIG. 11 is aprocess flow diagram of Machine Sync logic to control the relativeposition between the combine and grain cart. S1100: Operator in tractorengages system by pressing button on GUI in tractor. Combine operatorcan activate system when tractor gets close enough to the combine. Oncethe system is active, the tractor will accept ground speed and steeringcommands from the combine.

S1101: A GPS receiver that is mounted on the tractor transmits GPSlocation coordinates on tractor CAN bus. A controller on the tractor CANbus that is connected to a wireless communications transceiver reads thetractor motion dynamics (GPS location, GPS heading, and yaw (turn) rate)from the CAN bus and transmits the motion dynamics data using thewireless transmitter.

S1102: A controller on the combine that is connected to a wirelesscommunications transceiver receives the motion dynamics data transmittedfrom the tractor.

S1103: The same controller on the combine reads the combine motiondynamics information transmitted by a GPS receiver mounted on thecombine. The controller feeds the tractor and combine motion dynamicsinto a control algorithm.

S1104: The control algorithm takes the tractor and combine motiondynamics as inputs and outputs a ground speed and steering angle that issuitable for the tractor to position itself (or maintain its position ifalready suitable) relative to the combine such that the relativeposition of the tractor to the combine is suitable for cooperativeunloading while the combine continues to harvest.

S1105: The controller on the combine transmits the outputs of thecontrol algorithm using the wireless transmitter on the combine.

S1106: The controller on the tractor that is connected to a wirelesscommunications transceiver receives the recommended ground speed andsteering angle transmitted from the combine.

S1107: The controller on the tractor transmits on the CAN bus therecommended ground speed and steering angle to another controller on thetractor that is responsible for closed loop control of the ground speedand steering angle.

S1108: The controller on the tractor controls the tractor ground speedand steering angle to the commanded values.

In the present invention illustrated in FIGS. 1A and 3, a stereo camera(first imaging device 10) is mounted on the unloading auger 47 of acombine 2 looking in the direction a grain cart 4 pulled by a tractor 6(see FIG. 4) when the unloading auger 47 is rotated away from thechassis 7 of the combine 2. The stereo camera 10 is mounted sufficientlyhigh to have visibility into the grain cart 4 (see FIG. 2). This givesthe first imaging device 10 the ability to observe and profile thesurface 3 of the grain as the grain cart 4 fills. FIG. 1B illustratesthe first imaging device 12 on the spout of a forge harvester 31.

Additionally, another perception sensing device 12 (stereo camera—secondimaging device) is mounted on the chassis 7 of the combine 2 lookingdirectly to the left of the combine 2 (the side of the unloading auger13) as illustrated in FIG. 3. The chassis mounted stereo camera 12 isfurther from the grain cart 4 than the auger mounted sensor 10 duringunloading. This results in a better view of the grain cart 4 for thechassis mounted stereo camera 12 which facilitates easier tracking ofthe relative position of the grain cart 4 to the combine 2.

When the system is activated by the combine operator (button press onhydrohandle), the system determines if there is a grain cart 4 properlypositioned beneath the boot 13A of the auger so that no grain will bespilled when grain begins to flow. If a cart 4 is detected andwell-positioned, the system commands the combine 2 to turn on theunloading auger 47. The system continues to monitor the relativeposition between the combine 2 and grain cart 4 as well as the filllevel. As the grain cart 4 begins to become full, the system uses theprofile 3 of the grain surface to execute the operator-selected fillstrategy (back-to-front, front-to-back, front-to-back to front, etc.).In order to execute the fill strategy, the system needs to unload graininto the areas of the cart 4 that have less grain. The system can adjustthe unloading point by various means: command the auger 47 to rotate,command changes to the combine ground speed, command the combineoperator to manually adjust the combine speed, and/or command theMachine Sync system (discussed above and in U.S. Pat. No. 7,062,381 andU.S. Pat. No. 8,060,283, both incorporated by reference herein) tocommand the tractor 6 to change its relative position. When the entirecart 4 is filled to the level selected by the operator, the system turnsthe unloading auger 47 off.

Camera Orientation—

Tilting the optical axis (not shown) of the chassis mounted stereocamera 12 down 10-25 degrees from horizontal has several potentialbenefits. Firstly, less of the sky is in the field of view of the stereocamera. This helps to create a more uniform image intensity profile andmitigates potential dynamic range issues due to bright sunlight.Secondly, the bottom part of the grain cart 4 becomes more visible. Thisenables the stereo camera 12 to capture images of the grain cartwheel(s) 5. The wheel 5 is a feature on the grain cart 4 that can berobustly tracked by image processing techniques. Thirdly, tilting thestereo camera 12 down may mitigate the accumulation of dust and otherdebris on the lens or external window of the camera 14.

Sensing Auger Rotation—

If the combine 2 is equipped with a rotary position sensor (not shown)to measure the rotation angle φ of the unloading auger, it is possibleto fuse the data between the auger and chassis mounted stereo cameras10, 12. This enables the system to create a virtual profile of the grainlevel distribution inside the grain cart 4 even when the entire surface3 of the grain is not visible to the auger mounted stereo camera 10.Having this virtual profile of the entire surface 3 of the grain enablesthe system to intelligently execute a fill strategy for the grain cart(discussed above).

Relative Position Adjustment—

Generally, the system will attempt to utilize auger rotation φ as theprimary means of adjusting the area that grain is unloaded into thegrain cart 4. Utilization of proportional control valves (not shown) onthe hydraulic cylinder (not shown) that rotates the auger 47 facilitatesfiner adjustments to the auger position. The end result is a cart 4 thatis filled with a very even profile. Combines are typically equipped withhydraulic cylinders with valves that are not proportional. Systems oncombines 2 with non-proportional valves will generally fill the graincart with multiple discrete piles and have regions in the cart that arelocally high (above the desired fill level) and locally low (below thedesired fill level).

The default position for the auger 47 is at 90 degrees (perpendicular tothe centerline 21 of the combine 2) as shown in FIGS. 2 and 7. The augeron a conventional combine 2 is only permitted to rotate another 17degrees clockwise. However, the system is capable of operations up toany physical stop. The end 13B of the auger 47 traces an arc as itrotates about an axis of rotation (see FIG. 7). With these limitations,it is sometimes the case that a grain cart 4 cannot be filled using onlyauger rotation R. This is especially true for long grain carts 4.Therefore, the relative velocity between the combine 2 and grain cart 4must change in order to fill a grain cart 4 to capacity. FIG. 7 alsoillustrates a forward-to-aft zoned cart (zones 1-6) suitable for afront-to-back fill strategy. This arrangement of zones also includes“keep out” zones along the sides of cart 4 to minimize spillover of thecrop.

In particular, FIG. 7 is an illustration that shows how the lateraloffset between a combine and the trailer can govern the rotationallimits of the automated unloading system. The path that the tip of theboot of the unloading auger would follow as the auger is rotated fromits rest position to its maximum rotation angle is shown by the dashedarc (maximum rotational arc) about an axis of rotation. The trailer hasnon-fillable (keep out) areas along the inner perimeter of the trailer.The system will not unload material in these areas due to an increasedchance of spillage. The fillable area is the area between thenon-fillable areas extending from the front of the trailer to the rearof the trailer. The system divides the fillable area into some number ofzones (6 in this example) that are roughly the same size. The fillablearea and non-fillable areas are determine by the fill strategy.

In this example, the combine unloading auger is rotated out to 90degrees from its rest position. If the lateral offset and the fore/aftoffset between the combine and the trailer remain constant whileunloading, then the automated unloading system can unload material intozones 1, 2, 3, and 4. The system will set its lower rotational limit toan angle such that the tip of the boot of the auger remains over thefillable area of the trailer in the operation arc having a minimumrotational limit and a maximum rotational limit. The effective laterallength L_(lateral) of the spout 47 is equal to the length L_(a) of thespout 47 divided by the tangent of the elevation angle θ of the spout47. Thereby, one method to determine a rotational limit of the dischargeend of the spout within the maximum arc of the discharge end of thespout further takes into consideration at least one of the following: acurrent lateral offset between the storage portion and the transferringvehicle, a current height of the discharge end, the current rotationalangle φ of the spout, the current elevation angle θ of the spout, or aneffective lateral length of the spout.

Now consider a scenario where the fore/aft offset remains unchanged, andthe lateral offset between the combine and the trailer becomes slightlyshorter. The tip of the boot of the auger is now be positioned closer tothe far edge of the trailer. The lower rotational limit is a lowerangle, and the system could potentially unload into zone 5 as well.

Relative Position Adjustment by Controlling Combine Speed—

The speed of combine 2 can be controlled electronically when the combineis equipped with systems such as the Harvest Smart™ Feedrate ControlSystem. In the feedrate control system, the ground speed of the combine2 is automatically controlled by an electro hydraulic control valve (notshown). Ground speed is raised and lowered depending on the volume ofcrop that has entered the combine as sensed by a load pressure sensor(not shown) on the rotor (not shown) of the combine 2. To preventplugging the header 62 of the combine 2, the combine speed must notexceed the maximum speed allowed by the feedrate control systemnotwithstanding the ground speed that is commanded by the automaticunloading system.

Relative Position Adjustment by Integration with the Machine SyncSystem—

The speed of the tractor 6 pulling the grain cart 4 can be controlledvia the Machine Sync system. This would be the preferred method ofchanging relative position. When the automatic unloading systemdescribed herein is used in conjunction with Machine Sync, the combinedsystem would behave as follows:

1. The automatic unloading system determines that a relative positionadjustment is needed to fill in an area of the grain cart that is belowthe desired fill level.

2. The automatic unloading system sends a CAN message to the MachineSync system commanding that the tractor 6 change its position relativeto the combine 2.

3. The Machine Sync system on the combine 2 wirelessly transmits amessage to the tractor 6 commanding a change in relative position.

4. The Machine Sync system on the tractor controls its speed andsteering to execute the relative change in position.

Combining the automated unloading and Machine Sync systems couldpotentially automate the entire unloading process. The process stepsinclude:

1. A tractor 6 pulling a grain cart 4 approaches a combine 2, and theMachine Sync system controls the tractor 6 to a position that matchesthe speeds of the tractor 6 and combine 2.

2. The automated unloading system automatically rotates the auger 47 outfrom its rest position on the combine 2 and turns on the unloading auger47.

3. The automated unloading system executes the fill strategy and turnsthe auger 47 off when it reaches the desired fill level.

4. The Machine Sync system determines that the tractor 6 and grain cart4 have pulled away from the combine 2.

5. The automated unloading system retracts the auger 47 to the restposition.

The end of the auger raises (elevates) as it rotates clockwise andlowers (descends) as it rotates counter-clockwise. In order toautomatically extend or retract the auger 47, the system must ensurethat the relative position between the grain cart 4 and combine 2 issuch that the auger 47 will not make contact with the grain cart 4 as itrotates R. Any contact will likely result in damage and wear and tear onthe auger assembly.

One embodiment of the present invention calculates the height of the end13A or boot of auger 47 by the following equation:H _(b) =H _(r)+φ/90*L _(a)*tan θ

wherein H_(b)=height of the boot, H_(r)=height of the boot in restposition, φ=degrees auger has rotated from rest position, L_(a)=lengthof unloading auger, θ=rise angle of auger as it rotates.

Example

H_(r)=3 meters

L_(a)=6.9 meters

θ=8 degrees

The above equation is an example and any suitable equation can beincorporated into the present invention.

Another major benefit of integrating the automated unloading system withthe Machine Sync system is that the chassis-mounted stereo camera 12 isno longer required to track the relative position of the grain cart 4with combine 2. The Machine Sync system already tracks changes inrelative position by exchanging GPS coordinates via wirelesscommunications between the combine 2 and the tractor 6.

Relative Position Adjustment by Tractor Operator Notification onDisplay—

If the tractor 6 and combine 2 do not have the Machine Sync systemenabled, but they have requisite hardware for inter-vehiclecommunications, the automated unloading system could wirelessly transmitrelative position adjustment commands from the combine 2 to the tractor6. When the command is received by the display, the display could soundan audible signal to the tractor operator and display specificinstructions about the magnitude and direction of the adjustmentrequested by the automated unloading system.

Relative Position Adjustment by Combine Operator Notification onDisplay—

Another method of relative position adjustment is simply for theautomated unloading system to send a message to the combine display torequest a relative position adjustment. The display would then sound anaudible signal to the combine operator and display specific instructionsabout the magnitude and direction of the requested adjustment.

Target-Based (see FIG. 4) and Non-Target-Based Tracking—

The system is intended to track the relative position of the grain cartto the combine without the aid of fiducial markers (targets that areeasily identified through image processing). However, creating a systemthat works without the aid of targets is more difficult to achieve forseveral reasons. The first reason is that there is a lot of diversity inthe grain carts that the system may encounter in the field. Devising animage processing algorithm that works with all of carts is challenging.Secondly, some carts lack features that are useful for trackingpurposes. Thirdly, some carts are more difficult to track at nightwithout an exorbitant amount of illumination from the combine directedat the grain cart.

Cart Identification—

Besides providing easily identified tracking features or targetsdesigned to encode information regarding the identity of the grain cartor wagon, a cart or container identification module (as discussed aboveand in U.S. Pat. No. 8,649,940, incorporated herein by reference) can beincorporated into the system architecture to identify cart features suchas edges and sides. The cart identity could be tied to other information(grain moisture, seed variety, yield, etc.) that can be sensed by othersensors on the machine. Also, the cart identity could be used toautomatically load system settings such as the desired fill level andfill strategy (front to back, back to front, etc.)

Illumination for Nighttime Operation—

In order for the system to work at night, additional lighting must beprovided by the combine 2 to illuminate the grain cart 4 and make itvisible to the stereo cameras 10, 12. The lights should be mounted in aposition relative the cameras 10, 12 such that the backscattering of thelight into the camera lenses is minimized. This is achieved by puttingas much distance between the cameras 10, 12 and lights as possible andaiming the lights such that the direction they are pointed issignificantly different from the optical axis of the cameras.

User Interface—

The system could also feature the ability for the customer to view avideo feed from any of the cameras 10, 12 in the system on a display.The system could also send images to the display that are annotated bythe image processing software (disclosed above) to outline and highlightfeatures on the cart that the system is tracking.

Adjusting Relative Offset Between the Combine 2 and Grain Cart 4—

Now turning to FIG. 2, in order to achieve optimal system performance,it is beneficial to consider the auger rotation angle φ and the fore/aftoffset between the combine 2 and the grain cart 4. FIG. 5 illustratesthe method process steps for one embodiment of the present invention.

S500: The automated unloading system is activated by the operator bypressing a button on the hydrohandle. On the combine, the augerautomatically rotates out to its maximum rotation angle (and thereforeheight). On the SPFH the spout automatically rotates to predefinedlocation that is suitable for unloading to the left or right of themachine. The automated unloading system must then identify and starttracking a trailer prior to commencing active control.

S501: Once the automated unloading system is tracking a trailer, itgathers inputs such as the images from the stereo camera, the currentauger or spout rotation angle, static parameters (machine dimensions,mounting location and orientation of the stereo cameras, etc.) andoperator inputs to the graphical user interface (desired fill strategyand desired fill height). See FIG. 6 for image processing details.

S502: If on a combine, the system then makes calculations to determinethe rotational limits of the auger to avoid interference with thetrailer. This step is unnecessary on the SPFH, because the tilt and flapof the spout can be adjusted to avoid interference with the trailer, andthe SPFH propels the harvested material through the spout at a highvelocity. Instead, the rotational limits of the SPFH are predeterminedin the system software. On the combine there are 3 possible scenarios inwhich the auger can interfere with the trailer. The first scenario isthat the auger boot can interfere with the grain as it accumulates inthe trailer. If the grain piles up to the discharge point on the boot,damage can occur to the auger or the auger drive system. The equationsbelow will determine minimum auger boot height:H _(b) >H _(t) +H _(f)

-   -   H_(b)=height of the boot        −H _(b) =H _(r)+φ/90*L _(a)*tan θ    -   H_(t)=height of the trailer    -   H_(f)=height of the fill is used if the desired fill is to        exceed the height of the trailer, otherwise use 0.        H _(b) =H _(r)+φ/90*L _(a)*tan θ>H _(t) +H _(f)        φ>(90/(L _(a)*tan θ))*(H _(t) +H _(f) −H _(r))

The second scenario is that the auger boot can interfere with thetrailer edge as the auger rotates.H _(b) >H _(t)

-   -   H_(b)=height of the boot, where H_(b)=H_(r)+φ/90*L_(a)*tan θ    -   H_(t)=height of the trailer        H _(b) =H _(r)+φ/90*L _(a)*tan θ>H _(t)        φ>(90/(L _(a)*tan θ))*(H _(t) −H _(r))

The third scenario is that the auger or spout 47 can interfere with orcontact the top edge 4A of the trailer or storage portion 4 asillustrated in FIG. 7A. To determine an appropriate lateral offset toassure no contact between the auger or spout 47 with the top edge 4A oftrailer or storage portion 4, the lateral offset must be greater thanthe vertical distance B from a spout rotational pivot point P to the topedge 4A of the storage portion (B=H_(t)−H_(p), where H_(t) is the heightof the top edge of the trailer 4 above the ground, H_(p) is the heightof the spout/auger rotational pivot above the ground) divided by thetangent of the current elevation angle θ of the spout (minimum lateraloffset). The processor executes the following steps to assure that thereis no interference or contact between the auger/spout 47 and the topedge of the trailer:

determining a vertical distance from a spout rotational pivot point tothe top edge of the storage portion;

calculating a tangent of the current elevation angle θ of the spout;

dividing the vertical distance by the tangent of the current elevationangle θ of the spout to calculate a minimum lateral offset; and

adjusting the current lateral offset to a subsequent lateral offset whenthe current lateral offset is greater than the minimum lateral offset.

First, the system uses stereo vision ranging to determine the relativelocation of the trailer to the combine, the trailer height, and thetrailer length. The system calculates the rotational limits of the augerusing static parameters, the desired fill height (if it is set above thetop plane of the trailer), the trailer height, and the relative distanceof the trailer to the combine.

S503: The system then compares the current auger or spout rotation angleto the rotational limits. If the auger or spout is determined to be ator near the rotational limits, the process proceeds to the next step. Ifnot, the system loops back to gather inputs and the auger rotationallimits are recalculated again. It is necessary to periodically evaluatethe rotational limits, because changes in terrain could change theheight of the trailer relative to the combine or SPFH or the lateraloffset of the trailer may change. On the combine, changes in lateraloffset can affect the rotational limits, because the auger rotates in anarc and the harvested material is propelled downward at a low velocity.

S504: The system then determines if the harvested material is beingunloaded into the front most or rearmost zone of the cart.

S505: Once the system determines that harvested material is beingunloaded into the front most or rearmost zone of the cart, the systemdetermines if it can rotate towards the center of the cart withoutexceeding the rotational limits. If the system cannot rotate towards thecenter without exceeding the rotational limits, there is an increasedchance of grain spillage if the relative velocity of the trailer changessuddenly. For example, consider the scenario in which the system isfilling the front most zone of the cart and the spout or auger isrotated to its lower rotational limit. If the ground speed of thetrailer suddenly decreases, harvested material will over the front edgeof the trailer, and the system can only turn off the auger drive on thecombine to limit the spillage. However, if the auger or spout is not atits lower rotational limit as it unloads into the front most zone andthe ground speed of the trailer suddenly increases, the system canrotate the auger or spout towards the rear of the trailer to mitigate orprevent spillage.

S506: The system then considers the desired fill strategy to determinethe next zone in the trailer that the system will unload into when thecurrent zone reaches the desired fill level. The system calculates theauger or spout angle required to unload into the next fill zone. If theangle required to unload into the next zone violates the rotationallimits, the system will not be able to rotate the spout when the zonethat it is currently unloading into becomes filled to the desired filllevel. On the combine, the system will be forced to turn off the augerdrive when this occurs. On the SPFH, the system will overfill the zonethat is currently being unloaded into until the next zone can be reachedwithout violating the rotational limits of the spout.

S507: The system then takes measure to adjust the relative fore/aftoffset of the combine or SPFH to the trailer. If Machine Sync ispresent, the combine or SPFH can command the tractor to temporarilychange its ground speed until the next fill zone is within therotational limits or, in the case the that the system is unloading intothe front most or rearmost zone of the trailer, the adjacent zone in thecart is within the rotational limits. If Machine Sync is not present,the system can adjust the ground speed of the combine or SPFH. IfMachine Sync is not present and the system is unable to control theground speed of the combine or the SPFH, the system can send a CANmessage to the graphical user interface to sound an alert to notify theoperator that the system has reached its rotational limits. The systemor operator will shut off the auger drive (S508) if the fill strategy iscomplete or other limits are violated and would result in spill over ifcontinued.

For instance, when executing a back-to-front fill strategy, if the auger47 is fully rotated to maximum extension and grain is being unloadedinto the center of the grain cart 4, the system will not be able toextend the auger's rotation φ any further to fill the front of the cart4. Instead, the system should adjust the fore/aft offset between thevehicles (via Machine Sync or combine speed control) such that the auger47 is not fully extended when filling the center of the grain cart.Auger rotation φ is more responsive than adjusting fore/aft offset, andit is important to be able to unload grain into another area of the cart4 as quickly as possible when the current area reaches its desired filllevel.

Another reason to consider the rotation angle φ and fore/aft offset isto ensure that grain spillage can be easily mitigated. If the auger isfully extended as grain is being unloaded in the rear of the grain cart4, and the forward velocity of the grain cart 4 is greater than thecombine 2, the only recourse the automated unloading system has tomitigate grain spillage is to shut off the unloading auger 47. If theauger 47 is not fully rotated in this scenario, it can fully rotate theauger 47 as it shuts off the flow to further reduce the chances of grainspillage.

A stereo camera (not shown) could also be mounted on the cab of atractor.

Overcoming Conditions that Diminish System Performance—

Occasionally, the system may lose its ability to track the relativeposition of the combine to the grain cart or sense fill level. Multiplecameras are used to improve the accuracy and robustness of cart trackingand fill measurement. Each camera can be positioned on the body(chassis) of the combine or forage harvester or spout/auger end tooptimize the system function for cart tracking and fill measurement.Data fusion algorithms are used to register and combine the output ofthe multiple cameras to product a single, accurate, and robustmeasurement of the cart position and fill level. All the information onthe cart positions are integrated using a filtering algorithm, such as aKalman filter, to produce the estimate on the cart position andorientation. While all the information on the fill level are integratedusing a model based filter to produce an accurate measurement of thefill level. One embodiment of the present invention includes a built-inswitchover to handle failure in one or more cameras. If one or morecameras fail and are disabled during operation, then the filtering andregistration algorithms automatically uses information only from theremaining camera or Machine Sync data, if Machine Sync is available. Thesame switchover functionality can also be used to handle occlusion thatblocks one or more camera views. Failure detection uses consistency inthe measurements from multiple cameras. Use of the cart trackinginformation to perform selective stereo processing for fill measurementand better real-time performance. FIG. 6 illustrates the method processsteps for one embodiment of the present invention.

S600: After the images are gathered in S501 of FIG. 5, the systemdetermines if Machine Sync present. The presence of Machine Sync isdetected by monitoring for specific Machine Sync messages on the CANbus. If so, then the process continues to S602. Else, the system willreturn to S502 of FIG. 5 to make calculations using stereo visionranging to determine relative trailer location (S601).

S602: Gather Machine Sync offsets between combine or SPFH to tractor.

S603: Machine Sync calculates the relative location of the GPS receiveron the tractor to the location of the GPS receiver on the combine orSPFH. Since the location of the GPS receiver on the combine or SPFH isknown, the relative offset between the GPS receiver on the tractor andthe combine can be translated to a reference point on the vehicle (e.g.the center of the rear axle). The automated unloading system firstcalculates the relative location of the corners of the trailer to thestereo camera. Since the location of the camera on the combine or SPFHis known, the relative location of the trailer corners can be translatedto a reference point on the vehicle (e.g. the center of the rear axle).Once the location of the GPS receiver on the tractor and the corners ofthe trailer are translated to the same reference frame, the offsetbetween the GPS receiver and the trailer corners can be calculated.

S604: There are myriad sensor fusion possibilities when the Machine Syncoffsets and the automated unloading system offsets are translated to thesame coordinate frame. Here are a few: 1. The system could simply usethe offset from the data source that has the higher confidence. MachineSync could have low confidence when the GPS receiver is tracking fewsatellites, when GPS signal strength is weak, or interference preventsthe wireless radios from communicating. The automated unloading systemcould have low confidence when the camera is dirty, there is dust in theair, or the trailer is a long distance from the camera (stereo visionranging precision degrades quadratically with distance). For example,Machine Sync confidence could be the product of the GPS signal strengthtime of the combine (0-100%) times the GPS signal strength of thetractor (0-100%) times the signal integrity of the communications link(0-100%). The confidence of the vision processing, portion of the systemcould be the percentage of features that the system is able to trackfrom the previous image frame to the current image frame or related. 2.The system could use the offsets calculated by the vision processingportion of the system as long as it has high confidence and revert tousing the Machine Sync offsets when vision processing confidence ispoor. 3. The offset estimates from Machine Sync and the visionprocessing portion of the system could be tightly fused in a Kalmanfilter. The Kalman filter will weight the contributions from MachineSync and the vision processing portion of the system based on thereported confidence and error model of each data source.

S605: Use fusion algorithm output to determine relative trailerlocation. The system will return to S502 of FIG. 5 to make calculationsusing the fused data.

Potential causes include:

Thick dust between the combine and grain cart make it difficult orimpossible for the grain tank (chassis) camera 12 to see identifyingfeatures on the grain cart. This condition can occur when harvestingwith a tailwind.

Bright sunlight from a setting sun shining directly into the grain tankcamera 12 saturates the camera imager.

Dirt or other residue builds up on the camera window.

In the event of thick dust, the grain tank camera 12 cannot see thegrain cart 4. If the automated unloading system cannot track the graincart 4, it must shut off the auger 47, and the operator must unloadmanually. The automated unloading system could detect a loss of trackingfrom the grain tank camera 12 and use the auger camera 10 to detectfeatures on the grain cart 4 and track those features until the graintank camera 12 is able to track again. The auger camera 10 is betterable to detect features on the grain cart 4, because it is closer to thegrain cart 4 and has less dust obscurant to see through.

In the event of bright sunlight from a setting sun, the grain tankcamera 12 could become saturated and lose the ability to track. Thesystem could also use the auger camera 10 to track features on the graincart 4 during such occurrences. The auger camera 10 is much less likelyto become saturated by sunlight, because it has a steeper downwardviewing angle than the grain tank camera 12.

If automated unloading system is used in conjunction with Machine Sync,the relative position between the combine 2 and the grain cart 4 can becalculated using GPS data from the combine 2 and tractor 6 pulling thegrain cart 4. This calculated relative position could be an input to aKalman filter with the relative position that is sensed by the graintank camera 12. In the event that the grain tank camera 12 iscompromised by dust or direct sunlight, the relative position calculatedfrom the GPS data could be weighted much more heavily in the Kalmanfilter.

The automated unloading system can also measure the amount of residue onthe camera window. The system periodically captures and analyzes animage for signs of residue build up on the lens. There are a number ofcommercially available image processing algorithms designed to computean image sharpness or blurriness metric that could be correlated to thedirtiness of the camera window. The measured dirtiness can then bereported to the operator on the user interface. The operator can thendecide if it is necessary to clean the window immediately or to wait fora more convenient opportunity.

While the disclosure has been described in detail and with reference tospecific embodiments thereof, it will be apparent to one skilled in theart that various changes and modifications can be made therein withoutdeparting from the spirit and scope of the embodiments. Thus, it isintended that the present disclosure cover the modifications andvariations of this disclosure provided they come within the scope of theappended claims and their equivalents.

What is claimed is:
 1. A system for facilitating transfer of agricultural material from a transferring vehicle to a receiving vehicle, the system comprising: a spout comprising: a length, a rotational end pivotally connected to the transfer vehicle, and a discharge end, a rotational actuator to rotate the spout about a vertical axis of rotation from a rest position to a maximum position along a maximum arc, a vertical actuator to vertically elevate and lower the discharge end between a rest position height to a maximum position height, one or more sensors to determine a current rotational angle φ of the spout and a current elevation angle θ of the spout, and an auger drive to transfer agricultural material from the transferring vehicle to the receiving vehicle through the spout; the receiving vehicle comprising a propelled portion for propelling the receiving vehicle and a storage portion for storing agricultural material, wherein the storage portion comprising a front side, a rear side, and two longitudinal sides joined at corners, wherein each side has a top edge; a first imaging device connected to the spout and facing towards the storage portion of the receiving vehicle, wherein the first imaging device collects first image data; a second imaging device connected to a chassis of the transferring vehicle and facing towards the storage portion of the receiving vehicle, wherein the second imaging device collects second image data; and an image processing module associated with the transferring vehicle, wherein the image processing module comprises a processor for executing software to position the discharge end of the spout with respect to the front side or the rear side or the top edge of the storage portion of the receiving vehicle based on a fill strategy using the first image data and the second image data.
 2. The system according to claim 1, wherein the transferring vehicle and the receiving vehicle each comprise a location determining receiver and a wireless communication device, and wherein the processor further processes data from the receiving vehicle location determining receiver transmitted between the wireless communication devices, and data from the transferring vehicle location determining receiver to determine current offsets.
 3. The system according to claim 2, wherein the processor further executes the step of fusing the current forward-to-aft offset and the current lateral offset with the first image data and the second image data to calculate a relative location of corners of the storage portion.
 4. The system according to claim 3, wherein the processor further executes the step of determining a relative location of the storage portion.
 5. The system according to claim 2, wherein the processor executes the step of determining a rotational limit of the discharge end of the spout within the maximum arc of the discharge end of the spout.
 6. The system according to claim 5, wherein the rotational limit is defined as a portion of the maximum arc traversing fillable zones defined by the fill strategy.
 7. The system according to claim 5, wherein the step of determining a rotational limit of the discharge end of the spout within the maximum arc of the discharge end of the spout further comprises the step of determining at least one of the following a current lateral offset between the storage portion and the transferring vehicle, a current height of the discharge end, the current rotational angle φ of the spout, the current elevation angle θ of the spout, or an effective lateral length of the spout.
 8. The system according to claim 7, wherein the step of determining a rotational limit of the discharge end of the spout within the maximum arc of the discharge end of the spout further comprises the step of determining an interference point along the length of the spout with the top edge of the storage portion.
 9. The system according to claim 8, wherein the step of determining an interference point of the spout and the top edge of the storage portion comprises the steps of: determining a vertical distance from a spout rotational pivot point to the top edge of the storage portion; calculating a tangent of the current elevation angle θ of the spout; dividing the vertical distance by the tangent of the current elevation angle θ of the spout to calculate a minimum lateral offset; and adjusting the current lateral offset to a subsequent lateral offset when the current lateral offset is greater than the minimum lateral offset.
 10. The system according to claim 5, wherein the processor further executes the step of determining whether the discharge end of the spout is close to the rotational limit.
 11. The system according to claim 10, wherein the processor further executes the step of determining whether the discharge end of the spout is filling in the front most or rear most fillable zone.
 12. The system according to claim 11, wherein the processor further executes the step of determining whether the discharge end of the spout is able to rotate to one or more adjacent fillable zones.
 13. The system according to claim 12, wherein the processor further executes the step of adjusting the current forward-to-aft offset to a subsequent forward-to-aft offset when the spout is able to rotate to one or more adjacent fillable zones.
 14. The system according to claim 12, wherein the processor further executes the step of adjusting the current forward-to-aft offset to a subsequent forward-to-aft offset when the spout is not able to rotate to one or more adjacent fill zones.
 15. The system according to claim 1 wherein the first imaging device and the second imaging device each comprise a stereo vision camera.
 16. The system according to claim 1, wherein the processor executes the step of determining a rotational limit of the discharge end of the spout within the maximum arc of the discharge end of the spout.
 17. The system according to claim 16, wherein the rotational limit is defined as a portion of the maximum arc traversing fillable zones defined by the fill strategy.
 18. The system according to claim 16, wherein the step of determining a rotational limit of the discharge end of the spout within the maximum arc of the discharge end of the spout further comprises the step of determining at least one of the following a current lateral offset between the storage portion and the transferring vehicle, a current height of the discharge end, the current rotational angle φ of the spout, the current elevation angle θ of the spout, or an effective lateral length of the spout.
 19. The system according to claim 18, wherein the step of determining a rotational limit of the discharge end of the spout within the maximum arc of the discharge end of the spout further comprises the step of determining an interference point along the length of the spout with the top edge of the storage portion.
 20. The system according to claim 19, wherein the step of determining an interference point of the spout and the top edge of the storage portion comprises the steps of: determining a vertical distance from a spout rotational pivot point to the top edge of the storage portion; calculating a tangent of the current elevation angle θ of the spout; dividing the vertical distance by the tangent of the current elevation angle θ of the spout to calculate a minimum lateral offset; and adjusting the current lateral offset to a subsequent lateral offset when the current lateral offset is greater than the minimum lateral offset.
 21. The system according to claim 16, wherein the processor further executes the step of determining whether the discharge end of the spout is close to the rotational limit.
 22. The system according to claim 21, wherein the processor further executes the step of determining whether the discharge end of the spout is filling in the front most or rear most fillable zone.
 23. The system according to claim 22, wherein the processor further executes the step of determining whether the discharge end of the spout is able to rotate to one or more adjacent fillable zones.
 24. The system according to claim 23, wherein the processor further executes the step of adjusting the current forward-to-aft offset to a subsequent forward-to-aft offset when the spout is able to rotate to one or more adjacent fillable zones.
 25. The system according to claim 23, wherein the processor further executes the step of adjusting the current forward-to-aft offset to a subsequent forward-to-aft offset when the spout is not able to rotate to one or more adjacent fillable zones. 